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Starting with the second semester of the 1971-72 
school year^ the Phoenix Union High School System accepted a 
performance contract with Educators Service Incorporated of Edina^ 
Minnesota. The contract enabled the district to implement a 
one-semester program^ which was a multimedia course consisting of 36 
thirty-minute videotaped sessions which were divided into eight skill 
progression phases: initial testing^ preview skilly recall skilly 
learning techniques^ re-focus and review evaluation r memory skills^ 
vocabulary and depth reading skills^ and achievement tests and 
evaluation. The performance contract gueoranteed certain increase in 
reading rate and reading comprehension. These guarantees were not^ 
entirely met^ but there was significant improvement in these reading 
skills^ and the evaluation report recommended the purchase of the 
videotapes used. (Author/BW) 
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INTRODUCTION 

Starting with the second semester of the 1971-72 school year, the Phoenix 
Union High School System accepted a performance contract with Educators 
Service Incorporated of Edina, Minnesota (Appendix A). The contract 
enabled the District to implement a Visual Concepts Perceptive Reading 
Development and Applied Study Skills Course at Alhambra, Camelback, and 
East. 

The one-semester program was a multi*media course consisting of 36 thirty- 
minute video taped sessions which were divided into eight skill progression 
phases: 1) initial testing, 2) preview skill, 3) recall skill, 4)' learning 
techniques, 5) re-focus and review evaluation, 6) memory skills, 7) vocabulary 
and depth reading skills, 8) achievement tests and evaluation. The phases 
were sequential and cumulative, and structured on a concept of programming 
reading skills for video teaching and learning. It was hoped that moti- 
vation and individual achievement would be triggered through the correlation 
of video lessons i^lth jthe readi'ngf' techniques book, five modem paperbacks 
and audio reinforcement records for added drill and skill practice. 

District teachers conducted the video course after being given a pre- 
service workshop. 

HYPOTHESES ; 

The performance contract with Educators Service Incorporated guaranteed 
results as follows: 

1. The Reading Efficiency Index of students who have been given reading 
instruction via the video tapes will increase 150%. (Reading 
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Efficiency Index equals rate times comprehension). 
2. Class mean comprehension of the experimental group will improve 
a minimum of 10 percentile points if pre* test shows class mean 
is below the 50th percentile; class mean comprehension will 
maintain or improve if pre* test shows class mean at or above the 
50th percentile. 

3« Class mean reading of the experimental group will increase 30 
percentile points if pre*test scores show class mean below the 
60th percentile; class mean reading ratio will finish at or above 
the 95th percentile on the post-test if pre-test class mean was 
at the 60th percentile or above. 

4. Class mean on the total score of the Nelson-Demqr Reading Test 
will be statistically significant at the .05 level of confidence 
over the class mean in college reading, taugjht by a District 
reading teacher at Maryvale High School using District reading 
materials. 

THE MODIFIED PERFORMANCE CONTRACT ; 

In most performance contracts the coi]q>anyy not the district exercises 
control over teaching procedures , employment of staff , etc. The contract 
with Educators Service Incorporated allowed the district to choose the 
physical location, student population, and professional staff. The district 
agreed 9 however, to use the video tapes and purchased the software for 
$49000. It further agreed to purchase the tapes ($7,000) if all 4 hypotheses 
were verified.. 

THE EVAEUATION DESIGN ; 

Basically, the design called for a pre-test * post*test, experimental vs 
control group design. It can be depicted as follows : 



Experimental group 0^x02 ^ 

Control group 0^ O2 

Where Oi « pre test; O2 " post test; and X ■ the effect of the video tapes 
(treatment variable). 

The following students were identified as the experimental group: Four 
sections of reading students at Alhambra, five sections at Camelback, 
and four sections at East. The control group consisted of five reading 
classes at Maryvale. These students, of course, did not receive any 
instruction via the video tapes. 

The criterion variables (reading comprehension, reading rate, and reading 
efficiency - defined as reading rate times coiiq>rehension) were measured by 
the Nelson Denny Reading Test, both forms A and B, and the Einstein Test 
which is a part of the Visual Concepts television tapes. Mean scores were 
con;>ared for each group, pre and post. The influence of the independent 
variable (video tapes plus software vs no video tapes) upon the criterion 
(dependent. variables) was determined. 

Several "t" tests were used (1) to determine where the groups stood 
initially on the criterion measures (reading comprehension and reading 
rate) and (2) to determine statistically significant differences on the 
post'-'test. 

A measurement problem ensued, however, which prevented the use of a "t*' 
test for dependent sauries (one group coinpari son from pre t^ ^ 
two forms of the Nelson Denny Test (A and B) were not equivalent in terms 
of difficulty. For exaiq>le, a person scoring 34 on the test for reading 
coiq>rehension is at the 50th percentile oh form A9 but at the 54th 
percentile on form B. Therefore, it was Impossible to determine gains made 



from the pre to post test for one group without a standard score conversion, 
which the test publisher does not provide. 

There was away to skirt this measurement. ij^a^ue, however, and still judge 
the relative gains made by the two groups, and that is by using the 
Analysis of Covariance. A further advantage in using this statistical 
technique Is to equate differences that may have existed on the pre test, 
or some other variable such as ability. 

For exan^le, if 'the post-test was more difficult than the pretest (which 
it was in this case) and if the experimental treatment was effective, the 
relationship between the two groups from pre to post may have looked like 
the following: 

Pre Post 
Experimental ■ ■ ■ ■ ^ ^- i - i ^ ■ J 

Control Pre. 

Post 

If we were to use a "t'^ test (dependent saxnples) for the experimental group, 

no gain would be evident, but in our case that conclusion would be confused 

with the differences in the difficulty of the two forms of the test. 

However, the Malysis of Covariance would pick up the relative differences 

and the same "F'* ratio vould result for the above pattern as for the 

pattern below, where the tests are more parallel in difficulty. 

?ost 





Experimental Pre 
Control Pre Post 



The result would be the same since the Analysis of Covariance technique 
conqiares the regression lines of the two groups, thus measuring the relative 
gains (or losses) of the two groups. 

As was mentioned earlier, the Analysis of Covariance technique also coriti;pls ^or; 
dlffeTCnces which may be naturally present on some;^y^ 



the experiment. What are some natural variables to look at? One of course 

would be the California Test of Mental Maturity. The experimental group 

possessed a mean I.Q. of 113» while the mean for the control group was II69 

not far different, particularly considering the measurement error In ability testing 

Grade level may have also been a factor slcne lOth, llth, and 12th grade 
students were in each group. However, the mean grade level for the experi- 
mental group was 11.22 and 11.26 for the control group, again not much dif- 
ference. 

Significant differences did occur, however, on the pre^tests, both in read- 
ing cocqprehension and reading rate. It was decided then to use these two 
variables as the covarlates. To Insure that the treatment (x) had a chance 
for measured effect, any student In the experimental group who missed two 
consecutive tapes or three total tapes during the semester was eliminated 
in the analysis of the data. In the control group, any student who missed 
four consecutive days or six total days in the semester and who did not 
make up the work was likewise eliminated. Included in the analysis were 
those students with pre and post test data. 

THE DATA 

To test hypotheses #1 (Reading efficiency rate will Increase 150%) the 
means (pre and post) were compared as follows. 

TABLE 1 

••■ Gains- In -Reading Efficiency- - ...^..^,.„;^i^J^v. 




There was a great Increase in reading efficiency, particularly since reading 
efficiency is defined as reading rate times comprehension and particularly 
since the increase of 353.26 represents an increase in the average score^ 
not Just for a few individuals. Even though the mean score more than 
doubled^ the first hypothesis was not verified. The* experimental group 
fell short of its goal by 1«73 percent. 

To test the second hypotheses , the following data were collected and analyzed 
as shown in Table #2. 



TABLE ± 



Reading Comprehension Gains 







PRE 


-TEST 


POST~TEST 




N 


Mean 


(S.D.) 7.tile* 


N Mean (S.D.) 


7.tile* 


Experimental Group 


354 


38.49 


11.25 62nd 


354 42.40 13.10 


73rd 


Control Group 


124 


42.80 


12.16 73rd 


124 47.87 12.86 


84th 


"t" teats 

(Independent 

Samples) 


t = 3.580 
(Comparing the 
two groups on 
the pre -test) 


Significant 
at .01 level 
of confidence 


t = 4.00 Significant 
(Comparing the at .01 level 

two groups on of confidence 

the post— test) 



* %tlle8 were interpolated between scores on 11th grade norms and further 
corrected to account for the fact that the mean grade level for the 
experimental group was 11.22 and 11.26 for the control. 
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Hypothesis number 2, then, was verified by the data (Table 2). The 
experimental group was above the 50th percentile (62nd) initially and the 
group Improved their mean ranking on the p08t«test by 11 percentile points 
(73rd percentile). 

Note that statistically significant differences existed between the two 
groups on the pre^test as well as the post-*tests (Table 2). Nbte also 
that both groups made significant gains from pre to post, even though 
the form of the test taken as a post was more difficult than the pre. 

Not part of the hypotheses as contract; but still of interest is the fact 
that the control group also made an eleven point gain (percentile) from pre 
to post (73rd to 84th) as is shown in Table 2. 

Hypotheses #3 was tested by conqparing pre and post tests for reading rate 
as is shown in Table 3. 
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TABLE 3 
Eteadlng Rate Gains 







PRE. 


-TEST 




POST- TEST 




_^ 


Mean 


(T- 
(S.D."> 


7,tile* 


N 


Mean 


(S.D.) %tile* 


Experimental Group 


358 


217.06 


69.55 


39th 


358 


676.53 


241.60 99th 


Control Group 


125 


271.99 


85.23 


63rd 


125 


453.39 


115.34 97th 


"t" tests 

(Independent 

Samples) 


t » 7.128 
(Coiiq)arlng the 
two groups on 
the pre-test) 


Significant at 
.01 level of 
confidence. 


t - 9.821 
(Comparing the 
two groups on 
the post—test) 


Significant at 
.01 level of 
confidence. 



* %tlle8 were Interpolated between scores on 11th grade norms and further 
corrected to account for the fact that the mean grade level of the experi- 
mental group was 11.22 and 11*26 for the control. 



Hypotheses number 3 was verified by the data (Table 3). The group mean 
or the pre-test was below the 60th percentile (39th). On the post*test, 
a score of 488 would have been at the 99th percentile, but the group 
mean was 676.53 . To put it mildly, the experimental group blew the lid off 
the top of the test, and gained not 30 percentile points, but 60. 

The experimental group scored significantly higher on the pre-test as 
well as the post (Table 3) than the control group. As a point of interest, 
the control group did very well also going from a pre<»test percentile of 63 
to the 97th percentile on the post. 

To test hypotheses number 4 the Analysis of Covariance was utilized. The 
relative gains made by each group as well as controlling for the obvious 
differences existing between the two groups on the pre-*tests were measured 
by this technique. The data are as follows: 
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TABLE 4 



Relative Gains in Reading 
Comprehension and Reading Rate 
by Experimental and Control Groups 





GROUP 


READING COMPREHENSION 


READING RATE 


Experimental Group 
Control Group 


N "F" Ratio* Sienificance 


N "F" Ratio* Significance 


354 6.7 sig. at 

.05 level 

124 


358 145.6 sig. at 

.01 level 

125 





* as determined by Analysis of Covariance. 



Hypothesis #4, then was also verified. The experimental group gained relatively 
more in both reading comprehension and particularly reading rate when coiq»ared 
to the control group. 




CONCLUSIONS AND RECOMMENDATIONS 

!• With the exception of realizing an increase of 150% in reading 
efficiency (rate X coBq>rehension) , the video tapes and software 
supplied by Educators Service Incorporated reached or exceeded all 
expectations. Reading efficiency did increase by 148. 27%^ or only 
1.737. short of the goal. 

2. Both the experimental and control groups made significant gains in 
reading comprehension and reading rate, but the experimental group made 
relatively larger gains when controlling for differences that existed 
on the pre-* test. 

3. Reading rate gains were more significant than reading comprehensions 
gains. The experimental group reached the 99th percentile on the post- 
test as compared with the nationally normed groiq> after starting only 
four months before at the 39th percentile. 

4. It is recommended that even though the district is not coiq;>elled to 
purchase the video tapes, at a cost of $7,000, consideration be given 
to purchase them based on the results of this study. 



UfC:gc:ink 
7-12-72 
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APPENDIX A 
MODIFIED PERFORMANCE CONTRACT 
BETWEEN 

PHOENIX UNION HIGH SCHOOL DISTRICT AND EDUCATORS SERVICE INCORPORATED 

Educators Service Incorporated proposes to Improve the reading rate, compre- 
hension level and study skills of selected Phoenix Union Hl^h School System 
students through the implementation of the Visual Concepts Perceptive Read- 
ing Development and Applied Study Skills Course during second semester of 
the 1971-72 sc^iool year. > 

A* Educators Service Incorporated guarantees results as follows: 

1. The Reading Efficiency Index will increase 150% « The R.E.I, 
will be determined by computing the difference between pre and 
post test results of the Einstein test, based on class mean. 

2. Following results from the Nelson-Denny test Forms A ?^ the 
class mean comprehension difference between pre and post tests 
will be 

a. maintained or Improved if pre-test shows class mean at or 
above the 50th percentile. 

b. improved a minimum of 10 percentile points if pre-test shows 
class mean below 50 percentile. 

3. Following results from the NelflQni.Depnv test Forms A and B the 
class mean rejtdlny rate difference between pre aild post tests will 
be 

a. increased 30 percentile points if pre-test shows class mean 
below 60th percentile, 

b. at or above 95th percentile on post-test ifpte^test class 
mean was at the 60th percentile or above. 



4« Experimental group clas^ mean on the comprehension and reading 

rate score of the Nelson-Denny Reading Test will be statistically 
significant at the .05 level of confidence over the class mean 
of the control group on the total score of the Nelson-Denny Read- 
ing test. 

The control group will consist of approximately 100 students tak« 
ing a college-bound reading course at Maryvale High School, The 
course will be taught by a District reading teacher using District 
reading materials. Appropriate adjustments for statistically 
equating the control and experimental groups will be made by the 
District Research and Planning Division using the Analysis of 
Covariance Technique. 

Educators Service Incorporated agrees to provide the following: 
1. a pre*service workshop for staff and students in the program^ 
2« an in«*service workshop during the course for staff and students 
in the program, 

3« all course materials to include consumable workbooks, practice 
records, required paperbacks, test booklets, answer sheets and 
student Inventory forms ^ 

4« thirty six half*»hour video tape lessons for use during the 1971*»72 
school year at no charge. 

Educators Service Incorporated agrees that the Phoenix Union High 

School District will choose physical location, student population and 

professional staff. 

The Phoenix Union High School District agrees to the following: ^ 
1» the course will be given a minimum of two times a week in the 

second semester of the 1971 *-72 school year during regular school 

hours. 



2. the professional staff assigned to teach the course at the begin- 
ning of the second semester will be expected to remain with the 
same class throughout the duration of the course, barring Illness 
or other unexpected misfortune, 

3« Educators Service Incorporated representatives will be allowed 
to audit the course operation and offer suggestions, 

4. the Phoenix Union High School District will remit $4,000 for soft- 
ware to Educators Service Incorporated 30 days after the course Is 
initiated* This cost will not be refundable. 

Em Upon conclusion of the program the Phoenix Union High School District 
will 

1. purchase the 36 one-inch video tapes for $7,000 if post-test re- 
sults meet performance criteria. That amount will be due and pay- 
able 30 days after test results are submitted to the Phoenix Union 
High School District Board of Education, and after July 1, 1972, 

2 m return video tapes to Educators Sexrvlce Incorporated in good con^ 
ditlon If post test results do not meet performance criteria within 
30 days after final test results are submitted to the Board of 
Education, at no charge, 

3*' permit Educators Service Incorporated to use class mean pre and 
post test data for promotional purposes upon successful conclusion 
of the contract. 

' <} ■ 

I]fC:gc:ink ^ 
7-12-72 



